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REAL-TIME  IMAGING  FOR  CONSTRUCTION 
SITE  METROLOGY 


1  INTRODUCTION 


Background 

The  U.S.  Army  Corps  of  Engineers  {USACE)  is  responsible  for  a  wide  range  of 
construction  functions  involving  buildings,  dams,  waterways,  aircraft  runways,  and  other 
projects.  For  each  type  of  construction,  there  is  a  need  within  USACE  as  well  as  in  the 
private  sector  for  improved  methods  by  which:  (1)  quality  can  be  assured  throughout  the 
construction  process,  (2)  the  degree  of  construction  progress  can  be  assessed  and  docu¬ 
mented,  and  (3)  the  performance  of  systems  and  materials  can  be  assessed  over  time  to 
aid  in  maintenance  decision-making.  Traditionally,  these  aspects  of  construction  proces¬ 
ses  have  been  addressed  empirically,  largely  through  visual  assessments  and  a  wide  range 
of  reporting  formats. 

In  an  effort  to  reduce  construction  costs  to  USACE,  quality  assurance  (QA)  person¬ 
nel  have  become  responsible  for  assuring  quality  on  an  increasingly  large  number  of 
technically  diverse  projects.  Although  QA  personnel's  time  on  projects  has  decreased 
with  the  growing  demand  on  their  time,  the  necessary  documentation  and  awareness  of 
project  construction  processes  has  not.  To  enable  QA  personnel  to  maintain  excellence 
in  field  work,  their  capabilities  need  to  be  enhanced  and  optimized  through  state-of-the- 
art  technology. 

One  method  of  optimization  is  to  gather  QA  information  over  some  time  span  by 
acquisition  and  analysis  of  images  gathered  using  various  cameras.  The  images  and 
analysis  can  then  be  reviewed  by  QA  personnel  when  time  permits.  These  captured 
images  could  be  stored  for  documentation  or  further  processed  through  special  tech¬ 
niques  to  produce  information  that  previously  relied  on  labor-intensive  methods.  For 
example,  an  image  could  be  processed  to  highlight  and  calculate  an  area  that  changed 
from  a  previous  image.  QA  personnel  could  then  use  their  time  to  interpret  the  informa¬ 
tion  already  processed  and  make  decisions  when  necessary.  This  approach  would  opti¬ 
mize  their  time  by  permitting  them  to  be  in  the  field  longer  and  freeing  them  from  time- 
consuming  data  analysis  which  could  be  performed  by  computers. 

Recent  advances  in  computer  technology  have  expanded  the  opportunities  for 
improving  and  enhancing  traditional  QA  methods.  The  U.S.  Army  Construction  Engineer¬ 
ing  Research  Laboratory  (USA-CERL)  is  exploring  an  image-processing  concept  called 
the  sequential  construction  analyzer  (SCA)  to  improve  quality  assurance,  assist  in  track¬ 
ing  construction  progress,  and  provide  data  for  maintenance  decision-making.  This  work 
is  part  of  USA-CERL's  long-term  research  into  real-time  methods  of  QA  at  construction 
sites.  The  SCA  is  a  computer-based  system  that  quantifies  and  stores  images,  such  as 
those  obtained  photographically  using  either  visible  or  thermal  emanations.  The  SCA's 
image-processing  capabilities  include  enhancement  or  modification  of  the  image  to 
improve  its  appearance  or  highlight  information;  measurement  of  image  elements;  classi¬ 
fication  or  matching  of  image  elements;  and  recognition  of  items  in  the  image. 


The  SCA  is  intended  to  enhance  QA  personnel's  capabilities  while  providing  a  way 
to  improve  documentation  and  tracking  of  construction  processes.  Although  the  proto¬ 
type  system  will  focus  on  QA  during  construction,  many  other  applications  are  likely  in 
the  future.  Potential  applications  range  from  planning  and  design  processes,  through 
construction,  to  the  operation  and  maintenance  (0<5cM)  stage  of  a  project. 


Objective 

The  objective  of  this  study  was  to  identify  the  capabilities  and  potential  applica¬ 
tions  of  image  processing  in  construction-related  areas  of  primary  interest  to  USACE: 
buildings,  construction  sites,  and  pavements.  The  emphasis  was  to  be  on  QA  activities, 
with  a  secondary  goal  of  outlining  an  approach  by  which  other  potential  applications  can 
be  identified. 


Approach 

The  approach  taken  by  USA-CERL  was  to: 

1.  Identify  military  QA  personnel  needs  that  might  benefit  from  a  technology  like 
the  SCA. 

2.  Establish  the  state  of  the  art  in  imaging  technology. 

3.  Begin  developing  a  prototype  SCA  to  establish:  resolution  capabilities  on  a 
microcomputer-based  system  using  different  imaging  techniques;  the  algorithms  neces¬ 
sary  to  perform  various  image  analysis  processes;  and  the  relative  time  and  computer 
memory  needed  to  perform  the  various  analyses  for  the  prototype  application  as  a  model 
for  future  versions  of  the  SCA. 


Scope 

Information  in  this  report  is  a  theoretical  discussion  of  the  possible  USACE  applica¬ 
tions  of  image  acquisition  and  analysis  throughout  the  construction  process.  USA-CERL 
has  developed  imaging  systems  suitable  for  field  work.  USA-CERL  does  not  have  a  pack¬ 
aged  system  called  an  SCA,  but  has  the  equipment  comprising  an  SCA  which  can  be  con¬ 
figured  in  various  combinations  to  accommodate  site-specific  applications.  Such  applica¬ 
tions  are  developed  by  first  analyzing  the  information/product  needed  from  the  SCA. 
This  information  is  matched  through  knowledge  of  the  state  of  the  art  to  the  technology 
available  for  forming  a  site-specific  SCA. 


Mode  of  Technology  Transfer 

This  report  will  serve  as  the  basic  reference  document  for  future  development  of 
QA  guide  specifications  for  USACE.  It  will  also  provide  background  in  developing  poten¬ 
tial  applications  of  imaging  technology  for  other  aspects  of  the  construction  process. 
This  report  and  demonstrations  of  the  technology  will  be  presented  to  the  Construction 
Industry  Institute's  Advanced  Technology  Institute  as  a  form  of  technology  transfer  to 
private  industry. 


2  IMAGE-PROCESSING  TECHNOLOGY 


This  chapter  presents  an  overview  of  the  goals  and  existing  technology  in  the  field 
of  image  processing.  Included  are  descriptions  of:  (1)  a  generic  image-processing  system 
that  uses  a  personal  computer  and  an  image-processing  or  graphics  board  and  (2)  some  of 
the  more  commonly  used  image -processing  algorithms. 

Image  processing  is  the  science  of  modifying  and  analyzing  pictures.  The  basic 
goals  of  image  processing  include  enhancement  or  modification  of  an  image  to  improve 
its  appearance  or  highlight  information,  measurement  of  image  elements,  classification 
or  matching  of  image  elements,  and  recognition  of  items  in  the  image. 

Image  processing  is  performed  using  step-by-step  procedures  called  algorithms. 
These  algorithms  are  often  implemented  using  computers  that  are  flexible  and  have 
relatively  low  processing  and  memory  costs.  The  algorithms  are  expressed  as,  and  be¬ 
come  nearly  synonymous  with,  programs  for  the  computer.  An  algorithm  can  specify 
operations  such  as  how  to  acquire  the  image.  Special  image-processing  hardware  often 
supplements  the  computer. 

The  SCA  will  rely  on  image-processing  techniques  to  analyze  and  transform  an 
image  into  useful  QA  information.  This  information  can  then  be  interpreted  by  humans. 
To  establish  the  capabilities  and  applications  of  the  SCA,  it  is  necessary  to  have  a  thor¬ 
ough  understanding  of  current  image-processing  technologies  and  how  they  can  be  com¬ 
bined  to  produce  a  useful,  functional  system. 


Terminology  for  Processing 

Before  discussing  the  details  of  image  processing,  it  is  important  to  provide  a 
frame  of  reference  with  respect  to  the  terminology  used  in  this  report.  Because  so  many 
branches  of  science  have  used  image  processing  independently,  people  have  invented 
many  different  terms  that  describe  the  same  ideas.  This  report  describes  applications  of 
computer-assisted  image  processing  in  construction  using  terminology  from  the  computer 
science  and  electrical  engineering  disciplines. 


Image-processing  Hardware 

A  minimal  generic  image-processing  system  (that  could  comprise  an  SCA)  consists 
of  a  camera,  a  digitizing  circuit,  an  image  memory  (a  section  of  computer  memory 
dedicated  to  holding  the  image),  a  computer  that  can  access  image  memory,  and  a  device 
that  can  display  the  contents  of  memory  (i.e.,  the  picture  being  processed).  Such  a 
system  acquires,  processes,  and  displays  monochrome  images. 

The  digitizing  circuit  places  a  camera  image  into  the  image  memory.  This  process 
involves  digitizing  a  video  frame  (or  other  image,  such  as  a  photograph)  and  breaking  it 
into  an  array  of  digital  intensity  values  called  pixels  (short  for  "picture  elements").  If 
the  image  is  already  represented  as  pixels,  as  in  a  computer  graphic,  image  acquisition 
consists  of  simply  moving  the  image  from  disk  to  the  image  memory. 


The  computer  reads  and  writes  information  in  image  memory.  The  display  device 
reads  image  memory  and  shows  a  representation  of  it.  An  entire  video  image  stored  in 
image  memory  is  often  called  a  "frame  buffer"  or  "frame  store."  The  most  common 
frame  size  is  derived  from  the  National  Television  Standard  for  Color  (NTSC)  which 
results  in  a  frame  size  of  512  by  480  pixels. 

Most  low-cost  computers  work  with  data  that  is  represented  by  8  bits  (binary 
digits).  In  the  explanations  to  follow,  assume  that  a  pixel  has  an  8-bit  byte,  which  means 
that  it  can  have  28  (256)  different  gray-level  intensities.  Let  0  represent  black  and  255 
represent  white  with  all  the  corresponding  shades  of  gray  in  the  range  0  to  255. 

The  computer  manipulates  the  pixels  in  the  image  memory.  The  display  device 
converts  the  processed  pixels  back  into  spatially  organized  image  intensities.  Inis  dis¬ 
play  device  is  usually  a  digital-to-analog  (D/A)  converter  that  drives  a  monochrome  or 
color  television  monitor. 


Classifying  Algorithms 

Image-processing  algorithms  can  be  classified  in  many  ways.  If  an  algorithm 
changes  a  pixel's  value,  it  is  called  a  "point  process."  If  the  algorithm  changes  a  pixel's 
value  based  on  the  values  of  (1)  that  pixel  and  (2)  neighboring  pixels,  it  is  called  an  "area 
process."  If  the  algorithm  changes  the  position  or  arrangement  of  the  pixels,  it  is  called 
a  "geometric  process."  Algorithms  that  change  pixel  values  based  on  comparing  two  or 
more  images  are  called  "frame  processes"  (because  individual  video  images  are  called 
"frames"). 

Image  measurement  makes  few  assumptions  about  what  items  are  in  the  picture. 
Classification  and  recognition  require  successively  more  knowledge  about  what  can 
appear  in  an  image.  For  example,  the  number  of  pixels  in  a  remote  sensing  (satellite) 
image  of  the  Eaith  having  a  certain  range  of  values  can  be  measured.  If  it  is  known  that 
wheat  corresponds  to  these  values,  the  image  can  be  classified  into  wheat  and  r.cnwheat 
areas.  If  the  machine  is  provided  with  knowledge  about  the  structure  of  wheat  fields,  it 
might  be  able  to  recognize  these  fields  in  the  image.  Color  could  also  be  used  to  high¬ 
light  the  recognized  areas. 

Additional  ways  of  classifying  image-processing  algorithms  include  image-based 
versus  symbolic  methods,  linear  versus  nonlinear,  and  the  knowledge  level  used.  Image- 
based  algorithms  transform  pixel  values  into  other  pixel  values  or  locations  using  numeri¬ 
cal  or  logical  operations,  whereas  symbolic  algorithms  manipulate  data  that  represents 
knowledge  about  groups  of  pixels. 

The  knowledge  level  used  in  an  algorithm  can  range  from  simple  assumptions  about 
the  physics  of  image  formation  to  specific  knowledge  about  possible  items  in  a  scene. 
The  above  example  of  wheat  field  measurement,  classification,  and  recognition  illus¬ 
trates  the  knowledge-level  dimension.  Algorithms  are  image-based,  require  minimal 
knowledge  about  the  image  contents  and,  therefore,  are  not  recognition  algorithms. 

Based  on  this  classification  framework  and  the  conceptual  hardware,  some  example 
algorithms  will  be  aescriued.  Tnesc  examples  are  not  necessarily  complete  or  efficient; 
they  are  intended  only  to  facilitate  an  understanding  of  the  image-processing  options  for 
this  technology. 
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Point  Processes 


A  point-process  algorithm  scans  through  the  image  area  using  the  pixel  value  at 
each  point  to  compute  a  new  value  at  that  point. 

Optical  Negative 

To  produce  the  photographic  negative  of  an  image,  the  computer  is  programmed  to 
take  each  pixel  value  f  (0  <  f  <  255)  and  replace  it  with  g  where  g  =  255  -  f. 

Brightness 

Point  processes  also  can  be  used  to  enhance  or  modify  pixel  values.  For  example, 
adding  40  to  each  pixel  value  brightens  the  image  and  could  improve  the  picture's  ap¬ 
pearance. 

If  the  pixel  value  and  its  location  are  used,  then  a  point  process  can  be  used  to 
correct  shading  or  smoothly  change  pixel  values  in  an  image  area.  Shading  is  an  image 
artifact  caused  by  slow  spatial  shifts  in  scene  lighting  or  camera  bias  and  sensitivity.  A 
point  process  that  computes  the  inverse  of  a  shading  function  can  eliminate  or  correct 
much  of  this  shading. 

By  smoothly  changing  the  pixel  values  in  an  area,  the  contrast  of  areas  can  be 
highlighted  or  adjusted.  This  function  can  produce  results  similar  to  the  photographic 
darkroom  techniques  of  burning  and  dodging  (methods  of  adjusting  the  contrast  locally). 

Histogram  Stretching 

To  calculate  an  "intensity  histogram,"  the  number  of  times  a  particular  pixel  value 
occurs  in  an  image  is  counted.  Using  conceptual  hardware  based  on  8-bit  pixels,  256 
pixel  values  are  possible.  The  image  can  be  scanned  and  the  number  of  times  a  given 
pixel  value  is  found  counted.  The  result  can  be  stored  as  one  entry  in  a  256-place  table. 
Such  a  table  is  called  a  "histogram." 

The  histogram  is  an  example  of  image  measurement.  Because  it  examines  a  single 
pixel  at  a  time,  the  histogram  can  also  be  classified  as  a  point  process  that  leaves  the 
pixel's  value  unchanged. 

Information  provided  by  the  histogram  is  useful  for  image  enhancement  and  classi¬ 
fication.  If  all  pixel  values  are  bunched  in  a  small  range  (making  the  picture  appear 
featureless),  this  information  can  be  used  to  improve  image  contrast.  Starting  at  inten¬ 
sity  0,  the  histogram  can  be  searched  for  the  first  pixel  value  with  more  than  a  specified 
number  of  pixel  counts,  say  30.  Next,  a  similar  search  is  performed  starting  at  the 
highest  pixel  value.  The  region  of  the  histogram  between  these  two  (low  and  high)  values 
accounts  for  most  of  the  pixels  in  the  image.  Then  a  point  process  is  set  up  that  sets 
pixel  values  below  the  low  value  to  0  and  above  the  high  value  to  255.  This  is  sometimes 
called  a  "histogram  clip."  The  pixels  with  intermediate  values  are  adjusted  so  that  they 
span  the  range  of  0  to  255. 

Histogram  stretching  is  a  simple  form  of  contrast  enhancement.  Note  that  the 
image  has  lost  some  information— the  pixel  values  below  the  low  value  and  above  the  high 
value  have  been  set  to  constants.  In  general,  image-processing  operations  lose  some 
information  in  return  for  selecting  or  accentuating  other  information. 
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Notice  that  the  histogram  stretching  algorithm  used  three  simpler  algorithms:  a 
histogram,  a  histogram  clip,  and  a  point  process.  Most  algorithms  are  compounds  of 
other  algorithms.  Therefore,  the  program  writer  must  know  which  algorithms  to  apply 
and  in  what  order  to  apply  them  to  reach  a  processing  goal. 

Pseudocoloring 

Pseudocoloring  of  a  monochrome  image  is  another  example  of  a  point  process.  In 
this  case,  the  pixel  value  is  the  argument  (input)  for  three  different  functions,  and  the 
output  of  these  functions  drives  the  red,  green,  and  blue  guns  of  a  color  monitor.  This 
process  allows  a  monochrome  image  to  be  colored  by  substituting  any  color  for  a  particu¬ 
lar  shade  of  gray.  Of  course,  using  8-bit  pixels,  only  256  colors  can  be  displayed  at  once. 


Area  Processes 

An  area  process  uses  information  from  a  neighborhood  of  pixels  to  modify  pixel 
values  or  assert  the  existence  of  some  property  of  the  neighborhood.  Area  processes  can 
generate  a  wide  range  of  effects:  spatial  filtering  (such  as  filtering  out  repeated  ele¬ 
ments),  changing  an  image's  structure,  or  "sharpening"  the  image's  appearance  by  accen¬ 
tuating  intensity  changes.  Other  effects  include:  finding  objects  by  matching  images, 
measuring  image  properties,  making  assertions  about  object  edges  in  the  image,  removing 
noise,  and  blurring  or  smoothing  the  image. 

Convolution  is  a  classic  image-processing  algorithm  commonly  used  for  spatial 
filtering  and  finding  image  features.  Despite  its  name,  convolution  is  not  difficult  to 
understand.  However,  since  it  is  computationally  expensive,  many  subtle  implementation 
issues  must  be  considered. 

The  convolution  operation  replaces  a  pixel's  value  with  the  sum  of  that  pixel's  value 
plus  those  of  its  neighbors,  each  weighted  (multiplied)  by  a  factor.  The  weighting  factors 
are  called  the  "convolution  kernel."  The  programmer  labels  the  image  points  p  (i,j)  and 
the  kernel  (weighting)  points  k  (x,y),  where  x  and  y  range  over  values  representing  the 
relative  placement  of  neighboring  pixels. 

Choose,  as  an  example,  a  neighborhood  consisting  of  a  pixel  plus  its  immediate 
eight  neighbors  (left,  right,  up,  down,  and  diagonals).  Label  the  center  pixel  as  p  (1,1). 
This  center  pixel  is  replaced  by  the  linear  sum  of  its  neighbors  times  their  respective 
weighting  factors. 

P  (1,1)  -  p(0,0)  x  k(0,0)  +  p(l,0)  x  k( 1 , 0)  +  p(2,0)  x  k(2,0) 

+  p(0, 1)  x  k(0, 1)  +  p(l,l)  x  k( 1, 1)  +  p(2, 1)  x  k(2,l) 

+  p(0,2)  x  k(0,2)  +  p(l,2)  x  k( 1 , 2)  +  p(2,2)  x  k(2,2) 

To  convolve  an  image,  this  operation  is  repeated  at  every  pixel  position  in  the 
image.  You  can  think  of  this  process  as  sliding  a  kernel  matrix  over  each  row  of  pixels  in 


the  image  matrix.  At  each  point,  simply  multiply  the  kernel  values  with  the  image  value 
"under"  it,  sum  the  result,  and  replace  the  pixel  at  the  center  of  the  kernel  with  that 
value.  The  equation  then  becomes: 


p(i,j)  «-  p(i-l,j-l)  x  k(0,0)  +  p(i,j-l)  x  k(l,0)  +  p(i  l,j-l)  x  k(2,0) 

+  p(i-l,j)  x  k(0,l)  +  p(i,j)  x  k(l,l)  +  p(i+l,j)  x  k(2,l) 

+  p(i-l,j+l)  x  k(0,2)  +  p(i,j+l)  x  k(l,2)  +  p(i+l,j  +  l)  x  k(2,2) 
Implementation  Issues 

There  are  several  interesting  issues  to  be  met  here:  Convolving  an  area  of  size  X 
by  Y  with  a  kernel  of  size  N  by  M  requires  X  x  Y  x  N  x  M  multiplications  and  additions. 
Thus,  a  512  by  480  image  with  a  3  by  3  kernel  requires  2,211,840  multiplication/addition 
operations;  this  process  can  take  a  long  time  on  a  computer  without  fast  multiplication 
hardware. 

If  the  kernel  is  scanned  over  the  image  and  replaces  only  the  value  under  the  center 
of  the  kernel  at  a  given  position,  what  happens  to  the  edges  of  the  image?  For  example, 
with  the  3  by  3  kernel,  a  1-pixel  border  (box)  is  left  around  the  image  where  pixels  are 
not  replaced.  The  convolution  will  always  leave  a  border  of  "garbage"  equal  to  half  the 
kernel  size  around  the  image.  This  border  of  garbage  may  be  ignored,  set  to  0,  or  have 
the  nearest  meaningful  value  copied  into  it. 

The  convolution  on  any  pixel  could  result  in  a  value  larger  than  can  be  heid  in  a 
pixel— possibly  as  large  as  the  number  of  kernel  elements  times  the  number  of  bits  in  a 
pixel.  In  the  generic  system  described  previously,  for  example,  that  would  be  3  x  3  x  256 
=  2304,  for  which  12  bits  per  pixel  would  be  needed— but  there  are  only  8  bits  per  pixel. 
Enough  accuracy  in  the  calculations  must  be  kept  to  allow  for  this  range.  The  convolu¬ 
tion  result  can  be  scaled  (e.g.,  divide  each  result  by  2)  if  it  is  to  go  back  into  the  image 
memory. 

in  a  related  issue,  kernel  values,  and  therefore  the  convolution  output,  can  be 
positive  or  negative.  Negative  intensity  is  mathematically  useful,  but  not  physically 
reasonable.  In  this  case,  negative  intensity  cannot  be  displayed.  The  option  to  modify 
the  convolution  output  might  be  desired  such  that  only  positive,  negative  (negated  to 
positive  values),  absolute  values,  or  signed  values  are  output.  This  also  means  that  an 
additional  sign  bit  must  be  kept  in  the  calculations.  Thus,  for  8-bit  pixels  and  a  3  by  3 
kernel,  12  bits  for  the  sum  and  an  additional  bit  for  the  sign  are  needed. 

Clearly,  convolution  itself  is  relatively  simple,  but  the  implementation  issues 
complicate  it.  This  condition,  unfortunately,  is  true  of  most  image-processing  algo¬ 
rithms.  For  example,  the  issues  of  internal  accuracy  and  what  to  do  at  the  edge  of  the 
image  appear  in  most  other  area  processes.  Any  useful  program  must  address  these 
issues  to  implement  and  use  the  algorithms  effectively. 

Matched  and  Spatial  Filters 

When  applying  convolution  to  a  problem  in  image  processing,  the  program  writer 
should  generally  think  of  it  as  either  a  matched  filter  or  a  spatial  filter.  In  a  matched 
filter,  the  convolution  kernel  is  essentially  a  small  image  of  what  is  to  be  amplified  or 
detected.  For  example,  suppose  vertical  edges  in  the  image  are  to  be  amplified.  An 
edge  is  represented  in  the  image  by  a  sudden  increase  or  decrease  in  image  intensity. 


Such  a  kernel  looks  somewhat  like  a  vertical  edge: 
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Note  the  effect  of  the  negative  values:  in  a  uniform  image  area,  where  all  pixel  values 
input  to  the  convolution  are  the  same,  the  convolution  output  will  be  0  (since  the  sum  of 
any  number  times  each  of  the  15  kernel  elements  is  0).  The  kernel  has  been  padded  with 
a  vertical  row  of  zeros  to  make  it  an  odd  size  in  both  directions.  This  changes  the  pro¬ 
perties  of  the  kernel  but  is  a  computational  convenience. 

A  similar  kernel  for  amplifying  horizontal  edges  would  look  like  this: 

-1-1-1  -1  -1 
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Larger  kernels  can  be  used  with  a  pattern  (e.g.,  for  the  letter  A)  to  detect  similar 
patterns  in  the  image.  In  this  case,  the  kernel  is  often  called  a  "template."  Detection 
usually  involves  amplification  of  the  desired  feature  followed  by  a  yes/no  decision  that 
asks,  "Is  the  result  above  or  below  a  certain  threshold  point?" 

The  other  view  of  convolution  is  that  it  performs  spatial  frequency  filtering.  In 
sound,  frequency  is  the  number  of  times  per  second  a  waveform  repeats.  In  images, 
spatial  frequency  is  the  concept  that  an  image  can  be  broken  down  into  a  series  of  sine 
and  cosine  waves  of  various  frequencies.  This  breakdown  can  be  done  using  a  fast 
Fourier  transform.  The  transform  should  be  done  both  horizontally  and  vertically  be¬ 
cause  there  are  spatial  frequencies  in  both  directions. 

To  select,  and  perhaps  detect,  a  certain  band  of  frequencies,  a  kernel  can  be  built 
that  selects  that  frequency  band.  Quickly  changing  image  intensities  are  represented  by 
high  spatial  frequencies,  whereas  slowly  changing  intensities  are  represented  by  lower 
spatial  frequencies.  To  select  high  spatial  frequencies,  the  following  kernel  could  be 
used: 

-1  -1  -1 
-1  8  -1 
-1  -1  -1 

This  is  often  called  a  "Laplacian  filter"  because  it  approximates  an  unoriented 
second-derivative  operation.  Because  edges  have  high  spatial  frequencies  (sudden  inten¬ 
sity  changes),  this  kernel  selects  edges  of  any  direction.  It  might  be  used  as  an  "edge 
detector"  for  image  analysis. 

If  the  Laplacian  kernel  is  slightly  modified  by  making  the  center  kernel  element  9 
instead  of  8,  the  result  is  the  same  as  if  the  output  of  the  Laplacian  convolution  were 
added  to  the  original  image  (since  a  kernel  with  a  1  in  the  middle  surrounded  by  zeros 
would  yield  the  source  image  unchanged).  This  kernel  selectively  boosts  high  frequencies 
(edges)  and  the  resulting  image  looks  sharper  and  noisier.  On  the  other  hand,  if  a  kernel 
were  used  that  matches  lower  spatial  frequencies,  the  image  will  be  blurred. 
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The  power  of  convolution  lies  in  using  information  in  an  area  to  make  assertions 
about  some  property  at  an  image  point.  For  example,  the  edge  operators  shown  above 
improve  the  estimate  of  "edginess"  at  an  image  point  by  using  the  fact  that  physical 
edges  extend  over  some  distance.  The  art  of  convolving  is  in  picking  the  right  kernel. 
Experience  and  theory  are  necessary  for  creative  convolving. 

Nonlinear  Area  Processes 

Convolution  is  relatively  easy  to  implement,  use,  and  analyze  because  it  is  a  linear 
operation:  it  requires  only  sums  of  first-degree  products.  Nonlinear  operations,  while  a 
bit  more  difficult,  are  also  useful  and  can  be  more  powerful  than  a  convolution.  "Power¬ 
ful"  means  that  they  provide  a  better  signal-to-noise  ratio  for  detecting  image  elements 
or  can  detect  features  with  less  computation.  Consider  the  following  two  examples  of 
nonlinear  area  processes. 

Sobel  Filtering 

A  Sobel  filter  compares  the  result  of  two  convolutions.  The  first  convolution 
computes  the  degree  to  which  an  edge  exists  oriented  in  the  x  direction,  the  second 
computes  the  same  for  the  y  direction.  Simple  trigonometry  is  used  to  estimate  the 
strength  and  orientation  of  edge.  The  two  kernels,  X  and  Y,  are: 

-10  1  12  1 
X:  -2  0  2  Y:  0  0  0 
-10  1  -1  -2  -1 

Thus,  the  edge  strength  and  orientation  are  represented  by: 

Strength  =  sqrt(X2  +  Y2) 

Orientation  =  arctan  (Y/X) 

This  is  a  first-derivative  (oriented)  edge  finder,  and  the  vector  field  it  produces 
cannot  be  directly  shown  on  a  two-dimensional  image.  The  Sobel  is  a  good  edge  detector 
and  is  frequently  used  as  the  first  step  in  machine-vision  algorithms.  Because  the  Sobel 
algorithm  is  computationally  intensive,  various  approximations  have  been  developed  to 
implement  it. 

Median  Filtering 

A  median  filter  replaces  the  pixel  at  the  center  of  a  neighborhood  of  pixels  with 
the  median  of  the  pixel  values.  The  neighborhood  values,  including  the  center  pixel,  are 
sorted  into  ascending  order  and  the  median  (middle)  value  is  used  to  replace  the  center 
pixel.  The  effect  of  a  median  filter  is  to  remove  spot  noise  and  low-level  noise  while 
retaining  larger  scale  image  features. 

Pattern  and  Object  Identification 

To  classify  image  elements,  the  computer  must  be  supplied  with  additional  know¬ 
ledge  as  to  what  constitutes  an  image  element.  This  knowledge  can  be  elaborate,  but  a 
simple  rule  will  suffice  as  an  example:  an  element  is  a  connected  group  of  pixels  with 
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the  same  value.  "Connected”  means  that  if  the  pixel  has  the  same  value  as  a  neighbor  at 
0,  90,  180,  or  270  degrees,  it  is  part  of  the  element  group. 


To  simplify  the  computation,  the  image  can  be  "binarized”;  that  is,  all  pixel  values 
below  a  threshold  value  to  0  and  all  those  above  the  threshold  value  to  255  can  be  con¬ 
verted.  This  is  often  done  in  machine  vision  since  it  can  control  the  lighting  and  objects 
to  be  viewed.  The  image  is  then  searched  from  top  to  bottom  in  order  to  find  areas  that 
have  more  than  N  connected  pixels.  By  requiring  that  the  area  have  more  than  N  pixels, 
the  small  spots  of  noise  introduced  by  thresholding  can  be  ignored.  Each  area  is  labeled 
by  changing  the  pixel  values  in  it  to  a  particular  pixel  value.  The  number  of  pixels  in  an 
element  can  also  be  recorded  and  used  for  further  classification. 


Geometric  Processes 

Geometric  processes  change  the  spatial  arrangement  of  pixels.  They  are  often  used 
to  correct  for  distortions  caused  by  camera  optics  or  viewpoint.  They  also  can  enlarge 
an  image  area  of  interest.  Typical  geometric  processes  rotate,  stretch,  and  translate  the 
image  position.  Other  geometric  processes  can  warp  the  image.  Geometric  algorithms 
can  be  expressed  by  a  set  of  equations  (a  matrix)  that  maps  a  pixel  at  location  x,y  into  a 
new  address  x',y'.  For  example,  to  rotate  a  square  area  clockwise  by  90  degrees,  the 
pixels  are  mapped  by:  x'  =  (512  -  1)  -  y  and  y'  =  x. 

Because  digital  pixels  are  oriented  in  a  strict  checkerboard  pattern,  most  geo¬ 
metric  transforms  are  found  to  have  gaps  between  the  output  pixels.  If  source  pixels  are 
placed  in  the  destination  area  according  to  the  transformation  equations,  the  exact 
placement  in  the  output  image  is  rarely  an  integer.  (This  is  one  problem  with  digitized 
images  that  never  occurs  in  continuous-tone  images  such  as  photographic  prints.) 


Frame  Processes 

Algorithms  that  use  more  than  one  image  are  sometimes  called  "frame  processes." 
A  simple  example  is  to  subtract  one  image  from  another.  The  resulting  differences  can 
be  used  to  compare  the  two  images  (e.g.,  to  look  for  missing  parts  on  a  machine  or  cir¬ 
cuit  board).  Frame  processes  can  also  be  used  to  improve  image  quality  and  detect 
motion. 

With  a  television  camera  to  view  a  static  image  (i.e.,  nothing  moving  in  the  image), 
N  successive  image  frames  can  be  summed  to  reduce  noise  introduced  by  the  camera. 
This  process  requires  a  frame  memory  with  enough  bits  per  pixel  to  accommodate  the 
sum.  Dividing  the  sum  by  N  produces  an  averaged  image.  If  the  noise  is  not  correlated 
from  frame  to  frame,  the  improvement  in  signal-to-noise  will  be  of  the  order  sqrt(N).  A 
typical  low-cost  video  camera  has  about  3  bits  of  noise,  so  that  averaging  with  an  N  of  8 
or  16  will  noticeably  improve  the  image. 


Construction- Related  Capabilities 

This  section  contains  a  broad  overview  of  the  image-processing  capabilities  and 
how  they  spring  from  the  basic  algorithms.  Knowledge  of  these  algorithms  and  capabili¬ 
ties  forms  the  basis  for  identifying  and  establishing  applications  where  this  technology  is 
practical  and  beneficial.  Chapter  3  explains  their  particular  usage  in  construction.  The 
definitions  given  here  are  intended  to  be  very  general;  comprehending  the  more  complex 
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processing  techniques  requires  an  understanding  of  many  other  theoretical  and  technical 
issues  (e.g.,  sampling  procedures,  lighting,  and  algorithm  stability)  and  is  beyond  the 
scope  of  this  preliminary  study. 

Skew 

The  Sobel  kernel's  sensitivity  to  edge  direction  gives  an  image-processing  system 
the  ability  to  determine  if  an  edge  is  (1)  straight  and  (2)  square  to  another  edge.  These 
characteristics  are  called  "skew." 

Documentation  of  Presence 

Using  frame  processes,  an  image-processing  system  can  determine  the  presence  or 
removal  of  an  object  in  a  scene.  By  using  statistical  methods  or  specialized  knowledge, 
it  may  also  be  able  to  determine  an  object's  absence,  even  if  it  was  never  present  in  a 
scene. 

Nonoptical  Images 

Image  information  can  be  gathered  from  sources  that  emit  energy  other  than  vis¬ 
ible  light.  Radar,  laser,  infrared,  and  other  energy  forms  can  yield  valuable  information. 

There  are  video  cameras  that  sense  temperature  (via  infrared)  rather  than  optical 
light.  An  image-processing  system  uses  this  information  to  analyze  temperature  and 
secondary  causes  of  temperature  change,  such  as  moisture,  electrical  resistance  to 
current,  and  exposure  to  sunlight. 

Measurement 

Edge  detection  quickly  isolates  various  objects  in  a  scene.  Using  other  techniques, 
these  objects  are  identified  and  measured  for  length  or  area.  In  this  way,  cracks,  voids, 
object  placement,  and  size  can  be  measured. 

Color 

Adding  color  capabilities  to  image  processing  generates  complicated  theoretical 
issues.  Color  image  processors  yield  useful  information  about  material  cleanliness, 
material  composition,  and  corrosion. 

Roughness 

Measuring  the  statistical  characteristics  of  surface  color  due  to  roughness  yields 
diagnosis  of  a  surprising  range  of  material  pathologies.  Using  this  technique,  researchers 
have  made  accurate  measurements  of  delamination,  coating  cohesion,  and  subsurface 
corrosion. 


3  SYSTEMATIC  ANALYSIS  OF  IMAGE-PROCESSING 
APPLICATIONS  IN  CONSTRUCTION 


There  are  several  ways  to  group  image-processing  tasks  that  would  be  useful  in  a 
construction  setting,  such  as:  visual  medium,  visual  function,  chronological  impact  on 
building  life  cycle,  and  building  subsystem.  To  present  a  comprehensive  overview,  each 
of  these  classifications  is  examined  and  some  possible  applications  of  image  processing  in 
construction  are  discussed. 


Classification  by  Visual  Medium 

Image  information  can  come  from  sources  other  than  light  in  the  visual  spectrum. 
Below  are  examples  of  nonvisual  information  that  can  be  supplied  to  construction  activi¬ 
ties. 

Ground-Penetrating  Radar 

New  types  of  radar  have  been  developed  that  can  provide  information  about  struc¬ 
tures  invisible  to  optical  survey.  Ground-penetrating  radar  can  detect  buried  objects  and 
identify  soil  characteristics. 

Thermography 

This  form  of  infrared  photography  is  useful  for  guessing  the  temperature  of  objects 
in  an  image.  By  inference,  it  yields  information  about  electrical  conductivity,  moisture 
content,  and  other  processes  that  can  induce  temperature  change. 

Laser  Distance  Scanning 

This  form  of  image,  similar  to  radar,  represents  the  distance  to  the  nearest  object 
for  each  point  in  an  image.  This  technique  could  prove  useful  in  monitoring  site  excava¬ 
tions. 

Laser  Interferometry 

This  form  of  image  information  also  uses  changes  in  distance.  Using  two  pictures 
of  the  same  object  under  slightly  different  conditions,  laser  interferometry,  because  it 
can  resolve  extremely  small  variations  between  the  two  images,  detects  extremely  small 
changes  in  distance.  It  can  detect  the  small  deflections  of  structures  under  varying 
loads.  Hence,  one  possible  application  is  in  evaluating  the  condition  of  existing  struc¬ 
tures. 


Classification  by  Visual  Function 

Based  on  the  techniques  listed  in  Chapter  2,  the  following  is  a  list  of  image-proces¬ 
sing  abilities  applicable  to  construction.  This  list  is  discussed  in  detail  later  in  this 
chapter. 

1.  Documentation  of  presence.  Determines  if  a  particular  object  is  present  in  an 
image.  Example:  has  the  electrical  conduit  been  installed? 
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2.  Thermal  performance.  Using  thermography,  affords  the  ability  to  assess  mate¬ 
rial  performance  via  temperature  measurement.  Example:  is  concrete  being  poured 
during  weather  that  is  too  cold? 

3.  Presence  of  moisture.  Using  thermography,  gives  the  ability  to  infer  moisture 
content  via  temperature.  Example:  have  the  roof  felts  been  allowed  to  become  wet? 

4.  Voids.  Gives  the  ability  to  assess  if  an  area  is  completely  covered  or  filled. 

5.  Cracks.  Detects  breakage. 

6.  Cleanliness.  Useful  in  assessing  if  materials  are  being  handled  correctly. 

7.  Topography.  A  synonym  for  roughness.  Sometimes  a  useful  measure  of  material 
properties  or  coating  quality. 


8.  Color.  Useful  for  assessing  material  handling  and  coatings. 

9.  Thickness.  The  ability  to  measure  thickness  is  useful  in  determining  adherence 
to  various  specifications. 

10.  Composition  type.  Gives  the  ability  to  infer  material  type  and  verify  its  correct 
usage. 

11.  Delamination.  Allows  you  to  detect  when  a  composite  material  is  separating. 

12.  Corrosion.  Gives  the  ability  to  detect  the  presence  of  corrosion. 

13.  Skew.  Shows  if  something  is  straight  and  square. 

14.  Electrical  conduction.  Via  thermography,  gives  the  ability  to  assess  if  electrical 
cabling  is  overheating,  thereby  implying  an  excessive  electrical  load. 

15.  Safety.  Certain  visually  detectable  events  imply  the  presence  of  safety  hazards 
(e.g.,  leaking  gas  or  oil). 

16.  Terminal  overheating.  Via  thermography,  offers  the  ability  to  assess  the  quality 
of  electrical  connections. 

17.  Size.  Allows  quantitative  measurement  of  length  or  surface  area. 


Classification  by  Chronological  Impact  on  Building  Life  Cycle 
Preconstruction  Site  Survey  and  Metrology 

Surveys  for  environmental  planning,  land  use  planning,  city/urban  planning,  map¬ 
ping,  and  site  survey  are  important,  generally  understood,  and  accepted.  If  the  survey 
process  is  defined  to  include  impact  of  construction  on  preexisting  infrastructure,  certain 
survey  capabilities  could  be  exploited. 


17 


! 

;  Information  gathered  via  ground-penetrating  radar  enhances  planning  by  document- 

|  ing  the  location  of  underground  pipes,  detecting  leaks  in  underground  pipes  and  storage 

1  tanks,  and  evaluating  structural  status  of  pavements  and  railroad  track. 

Laser  interferometry  is  used  to  measure  (1)  differential  settlement  of  bridges  and 
!  trusses  and  (2)  bridge  and  pavement  deflection  under  load.  Information  gathered  indi- 

|  cates  if  preexisting  structures  can  tolerate  increased  usage. 

Construction  in  Process 

The  SCA  is  being  developed  primarily  for  the  construction  process  although  it  could 
be  applied  to  many  other  areas.  The  SCA  on  a  construction  site  will  consist  of  one  or 
more  cameras,  strategically  located  to  observe  various  activities.  These  cameras  will 
feed  pictures  to  a  centralized  image  processor.  The  SCA  is  being  designed  to  perform  a 
range  of  tasks  without  user  intervention  and  to  keep  records  for  work  requiring  user  help 
or  other  ad  hoc  tasks. 

Detailed  lists  of  possible  SCA  applications  for  in-process  construction  are  given  in 
tables  at  the  end  of  this  chapter.  SCA  applications  during  construction  can  be  broadly 
grouped  as  triggers  for  critical  path  management  events,  quality  assurance,  and  supple¬ 
mental  data  for  construction  automation. 

When  used  for  critical  path  management,  SCA  assists  in  areas  of  poor  productiv¬ 
ity.  Construction  productivity  suffers  from  lack  of  proper  scheduling,  coordination  of 
tool  usage,  material  and  engineer  availability,  and  QA  scheduling.  A  finely  tuned  image- 
processing  algorithm  detects  construction  events  that  mandate  schedule  update  or 
change.  Examples  of  this  are: 

•  When  the  third  floor  electrical  wiring  goes  in,  schedule  an  appointment  with  the 

electrical  inspector. 

•  If  the  plumbing  is  not  installed  on  Tuesday,  alert  the  project  manager. 

QA  personnel  perform  a  wide  variety  of  tasks  ranging  from  assuring  correct  prac¬ 
tice  of  quality  control  to  judging  the  validity  of  contractor  claims  made  to  the  Govern¬ 
ment  for  unanticipated  costs.  To  do  this  work,  a  Qa  person  must  track  usage  of  heavy 
equipment  and  other  specialized  tools  (e.g.,  the  contractor  might  claim  the  costs  of  a 
crane  rental  on  a  crane  that  sits  idle).  By  using  ground-penetrating  techniques  with  an 
SCA,  information  such  as  presence  of  a  large  metal  structure  underground  can  be  estab¬ 
lished  prior  to  sending  out  a  contract  for  bid.  A  QA  person  could  then  schedule  an  in¬ 
spection  or  require  SCA  to  be  in  place  to  allow  for  verification  of  the  contractor's  work 
required  to  deal  with  the  structure.  Other  potential  performance  measurements  include: 

•  When  was  the  last  time  the  crane  moved? 

•  Are  the  supply  trucks  coming  in  fully  loaded? 

•  Have  the  roof  felts  been  allowed  to  become  wet? 

•  How  large  was  that  unexpected  boulder  excavated  last  week? 

•  What  percentage  of  this  metal  panel  is  corroded? 

•  Has  the  paint  been  applied  correctly? 
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•  Is  there  subsurface  corrosion  present  on  this  laminate? 

•  Is  that  beam  straight? 

As  construction  becomes  more  automated,  the  SCA  could  be  used  for  artificial 
vision.  This  application  would  have  two  principal  benefits:  augmenting  information 
available  to  these  automatic  systems  and  oversight  of  tasks  to  promote  safety. 

Benefits  After  Construction 

Information  retained  from  the  construction  process  would  asrrst  in  life-cyclc  main¬ 
tenance.  By  making  as-built  drawings,  recording  construction  techniques,  and  automat¬ 
ing  maintenance  management,  an  SCA  would  be  useful  after  construction  is  completed. 

Contractors,  usually  unenthusiastic  about  producing  as-built  drawings,  could  have 
this  requirement  semiautomated  by  feeding  pictures  of  the  building  (at  various  stages  in 
construction)  to  a  specialized  computer-aided  drafting  (CAD)  program.  This  program 
could,  with  some  human  help,  produce  accurate  drawings. 

As  with  every  form  of  industrial  art,  careful  planning  of  task  and  task  order  greatly 
reduces  cost  and  improves  results.  By  the  time  a  need  for  major  repair  of  a  structure 
arises,  some  of  the  techniques  used  in  its  construction  are  outdated  or  unknown.  This 
makes  repair  planning  difficult  and  uncertain.  Records  from  image  processing  could  be 
retained  on  compact,  stable,  and  easy-to-store  media  (such  as  compact  disks  or  video 
disks).  Access  to  these  records  would  make  repair  planning  easier  and  more  certain. 

Optical  records  of  construction  and  maintenance  activities  would  be  invaluable 
data  for  automated  maintenance  management  systems.  These  systems  require  a  compos¬ 
ite  index  of  structure  condition.  Such  indices  are  much  more  difficult  to  formulate 
without  detailed  knowledge  of  construction  techniques  (and  hence,  repair  techniques). 

Usage  of  automated  maintenance  management  techniques  has  been,  in  general, 
hindered  by  the  large  amount  of  time  invested  in  making  the  initial  assay  of  structures. 
Image  records  kept  during  the  construction  process  could  be  used  to  avoid  these  costly 
assays. 

Not  only  would  selected  construction  tasks  benefit  from  image-processing  applica¬ 
tions,  but  the  data  collected  from  these  applications  would  greatly  aid  life-cycle  mainte¬ 
nance.  Still  other  image-processing  applications  would  benefit  site  planning.  Because  of 
the  high-speed  image  acquisition  and  analysis  possible  with  new  and  emerging  microcom¬ 
puters,  real-time  or  quasi-real-time  evaluation  and  decision  support  are  possible. 


Classification  by  Building  Subsystem 

To  survey  potential  SCA  applications,  this  section  addresses  three  major  areas  of 
the  construction  industry  of  interest  to  USACE:  building  construction,  construction 
sites,  and  paving.  These  potential  applications  are  detailed  in  tables  that  (1)  divide  each 
construction  process  into  its  major  elements,  subelements,  and  materials,  and  (2)  identify 
major  properties  or  attributes  of  interest.  Within  each  table,  a  numbering  code  is  used  to 
identify  a  potential  SCA  application  and  the  most  likely  method  by  which  an  image 
portraying  the  application  can  be  generated.  For  example,  the  number  code  "l"  denotes 
the  use  of  a  video  camera  and  the  number  code  "2"  denotes  a  thermographic  camera. 


Buildings 


A  building  can  be  divided  into  five  major  elements  or  subsystems:  (1)  exterior 
envelope,  (2)  interior  space  divider,  (3)  electrical  subsystem,  (4)  plumbing  subsystem,  and 
(5)  space  heating  and  cooling  subsystem.  Each  of  these  elements  can  be  further  divided 
into  subelements  and  materials.  For  example,  subelements  of  the  exterior  envelope  are 
roof,  wall,  doors  and  windows,  foundation,  and  floor.  Applications  of  the  SCA  for  these 
five  major  elements  are  described  in  the  following  paragraphs  with  more  details  in  ac¬ 
companying  tables. 

Exterior  Envelope.  Table  1  lists,  on  the  vertical  axis,  the  subelements  and  mater¬ 
ials  for  the  exterior  envelope  of  a  building  and,  on  the  horizontal  axis,  key  system  and 
material  properties  or  attributes.  Major  subelements  of  the  exterior  envelope  are  those 
listed  above.  Properties  or  attributes  are  divided  into  system  and  material  properties. 
System  properties  could  be  assessed  by  the  SCA  on  the  completed  subelement  and  include 
s!:c’.v,  documentation  of  presence,  thermal  performance,  and  presence  of  moisture. 

To  illustrate  the  use  of  the  table,  examples  from  the  first  four  vertical  columns  of 
Table  1  are  cited.  Video  camera  photographs  (Code  Number  1)  can  be  used  to  document 
or  assess  skew  of  all  completed  exterior  envelope  subelements  (listed  as  "system"  under 
each  subelement)  and  components  of  some  subelements  (e.g.,  decking  and  structural 
members  of  the  roof).  Video  camera  photographs  can  also  be  used  to  document  the  pre¬ 
sence  of  all  subelements  and  other  components  or  materials.  This  use  of  photographs 
may  be  helpful  to  inspectors  in  ensuring  compliance  with  specifications  throughout  the 
construction  process.  Thermographic  photographs  are  likely  to  be  useful  in  assessing 
thermal  performance  of  completed  exterior  envelope  subelements,  such  as  the  roof,  wall 
and  doors,  and  windows. 

In  the  fourth  column,  titled,  "presence  of  moisture,"  both  video  and  thermographic 
photographs  may  be  useful  in  documenting  the  presence  of  moisture.  For  example,  mois¬ 
ture  on  or  in  the  completed  roof  could  be  detected  photographically  by  video  camera  if 
water  is  ponded  on  a  flat  roof  or  by  aerial  thermographic  photographs  if  the  presence  of 
water  is  not  so  obvious.  The  presence  of  excessive  moisture  in  insulation  of  roofs  and 
exterior  walls  might  best  be  detected  by  exposing  the  insulation  and  documenting  the 
moisture  with  a  video  camera. 

Interior  Space  Dividers.  Table  2  illustrates  potential  QA  applications  of  the  SCA 
for  subelements  and  components  or  materials  of  interior  space  dividers.  Major  subele¬ 
ments  of  interior  space  dividers  are  ceiling,  wall,  floor  and  stairs. 

As  for  the  subelements  and  materials  of  the  exterior  envelope,  video  camera  pho¬ 
tographs  can  be  used  to  document  the  presence  and  the  type  or  composition  (e.g.,  gypsum 
board,  tile  or  wood  ceiling;  clear  or  pigmented  coating)  of  all  subelements,  components 
and  materials  of  interior  space  dividers,  assuming  photographs  are  taken  while  items  of 
interest  are  visible.  Numerous  other  potential  applications  are  listed  in  Table  2. 

Electrical  Subsystem.  Table  3  shows  potential  SCA  applications  for  the  electrical 
subsystem  of  buildings.  Major  subelements  of  the  electrical  subsystem  include  (1)  service 
entrance,  (2)  branch  circuits,  and  (3)  other. 
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Table  1  (Cont'd.) 


Table  3 


Quality  Assurance  Applications  of  the  Sequential  Construction 
Analyzer  in  the  Electrical  Subsystems  of  Buildings 


PROPERTIES  OR  ATTRIBUTES' 


BUIlDING  ELEMENT 

SYSTEM 

PROPERTIES 

Documentat i on 

Condjct  of 

Safety 

ELECTRICAL  SUBSYSTEM 

of  presence 

E 1  ectr i c i t  y 

MATERIAL  PROPERTIES 


SERVICE  ENTRANCE 

Entrance  cable 

i 

1 

Breaker/fuse  box 

i 

- 

8reaker/f uses 

i 

- 

Fasteners 

i 

- 

Termi nat i ons 

i 

1 

BRANCH  CIRCUITS 

Cable 

i 

1 

Receptac  1  es 

i 

1 

Sw i tches 

i 

1 

Other  terminations 

i 

1 

Enclosures 

i 

1 

other 

Electrical  fixtures 

i 

1 

Electrical  app i i ances 

i 

1 

'Numbers  refer  to  the  most 

ikely  method  for 

generating  the  image 

to  be  analyzed  by  the  SCA : 

1 -v i deo  camera ; 

2  thermographic 

camera 

Video  camera  photographs  can  be  used  to  document  the  presence  and  the  type  or 
composition  of  all  components  or  materials.  For  example,  a  single  photograph  of  the 
service  entrance  cable,  including  its  label,  could  perhaps  be  used  to  document  its  pro 
sence,  the  type  of  cable  (aluminum,  copper),  its  size,  and  the  fact  that  it  has  been  termi¬ 
nated  in  compliance  with  specifications  (safety)  to  the  main  power  breaker. 

Under  the  property  "conduct  of  electricity,"  photography  is  not  identified  as  a 
direct  measurement  tool.  It  could,  however,  be  used  indirectly  to  document  the  flow  of 
electricity  by  photographing  an  operable  fixture,  such  as  a  light  bulb. 


Plumbing  Subsystem.  Table  4  presents  potential  SCA  applications  for  the  plumbing 
subsystem  of  buildings.  Major  subelements  of  the  plumbing  subsystem  include  transport, 
storage,  heating,  and  functional.  Video  camera  photographs  can  be  used  to  document  the 
presence,  type  or  composition,  and  size  of  all  components  and  materials.  For  compo¬ 
nents  of  the  Transport  Subelement,  the  applications  listed  in  Table  4  include  skew  or 
slope  of  piping  and  spigots.  Skew  or  slope  data  for  valves,  restraints,  and  fasteners  are 
unlikely  to  be  of  interest  and,  therefore,  are  not  shown  as  potential  applications. 

Space  Heating  and  Cooling  Subsystem.  Table  5  shows  potential  SCA  applications 
for  the  space  heating  and  cooling  subsystem.  Major  subelements  of  the  space  heating  and 
cooling  subsystem  include  (1)  generation  source,  (2)  transport,  (3)  storage,  and  (4)  other. 
System  properties  are  listed  as  skew  or  slope,  documentation  of  presence,  leaks,  and 
safety.  Examples  of  safety  concerns  for  generation  sources  that  could  be  addressed  using 
video  camera  photographs  are  distance  from  flammable  materials,  venting  facilities,  and 
restraints  or  fasteners. 

Construction  Sites 

Table  6  lists  potential  SCA  applications  at  construction  sites.  Elements  associated 
with  construction  sites  are  divided  into  three  categories:  below  grade,  ground  level,  and 
above  grade.  Properties  or  attributes  are  divided  into  two  main  categories:  topographi¬ 
cal  and  other.  The  SCA  could  be  used  at  construction  sites  to  document,  for  example, 
changes  in  terrain  as  a  result  of  the  construction,  changes  in  vegetation  patterns,  and 
slope  of  terrain. 

Paving 

Table  7  presents  potential  SCA  applications  in  paving.  Paving  elements  are  listed 
as:  (1)  construction  base,  (2)  pavement,  (3)  drainage  system,  and  (4)  safety-related  sys¬ 
tem.  Video  camera  photographs  can  be  used,  for  example,  to  document  or  assess  skew  or 
slope  of  completed  construction  bases,  pavements,  drainage  systems,  and  safety-related 
subelements  such  as  surface  markings,  signs  or  markers,  edge  grading,  and  guardrails. 
Likewise,  video  camera  photographs  can  be  used  to  document  the  presence  of  all  subele¬ 
ments,  as  shown  in  the  table. 

User  Survey  of  SCA  Application  Needs 

Figure  1  shows  the  results  from  surveys  conducted  during  the  Huntsville  Advanced 
Construction  Management  courses.  Other  field  personnel  were  surveyed  as  well.  The 
figure  shows  the  relative  frequency  of  responses  for  potential  applications  of  the  SCA  in 
the  categories  defined  by  Tables  1  through  6.  The  inspectors,  managers,  and  QA  person¬ 
nel  involved  in  the  course  listed  specific  applications  which  are  grouped  here  to  corre¬ 
spond  to  tables  presented  under  Buildings  above.  The  survey  results  demonstrate  field 
personnel's  perception  of  where  the  SCA  would  benefit  their  work.  In  general,  the  re¬ 
sponse  was  very  positive. 


Table  4 


Quality  Assurance  Applications  of  the  Sequential  Construction 
Analyzer  in  the  Plumbing  Subsystem  of  Buildings 


PROPERTIES  OR  ATTRIBUTES1 


BUILDING  ELEMENT 

SYSTEM  PROPERTIES 

PLUMBING  SUBSYSTEM 

Skew  or 

Slope 

Documentat ion 

of  presence 

Leaks  Corrosion 

Type  or 
Compos i t ion 

S  i  ze 

TRANSPORT 

Piping 

1 

1 

1  1,2 

1 

i 

Valves 

- 

I 

'  1  ,2 

1 

i 

Spigots 

1 

1 

1  1,2 

1 

i 

Res tra i nts 

- 

1 

1,2 

1 

i 

Fasteners 

- 

1 

1,2 

1 

i 

STORAGE 

Tank  ( inter ior) 

1 

1 

1 

1 

! 

S  i  nk 

1 

1 

1  1  ,2 

1 

1 

HEATING 

Water  heater 

1 

1 

1 

l 

1 

Solar  col lec torts) 

1 

1 

l  1 ,22 

1 

1 

FUNCTIONAL 

Tub 

Shower 

1 

1 

1 

1 

1  1,2 

1 

1 

1 

S  i  nk 

1 

1 

1  1  ,2 

l 

1 

Toi let 

1 

1 

’  1  ,2 

1 

1 

Laundry 

1 

1 

1  '  ,2 

l 

1 

D i shwasher 

1 

I 

1  1,2 

1 

1 

'Numbers  refer  to  the  most  likely  method  for  generating  the  image 
to  be  analyzed  by  the  SCA :  1-video  camera;  2-thermograph i c 
camera 

2 

Assuming  conduits  for  heat  transport  fluid  are  visible  from  the 
front  surface 
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Table  5 


Quality  Assurance  Applications  of  the  Sequential  Construction 
Analyzer  in  the  Space  Heating  and  Cooling  Subsystem  of  Buildings 


PROPERTIES  OR  ATTRIBUTES 


BUILDING  ELEMENT 

SYSTEM  PROPERTIES 

SPACE  HEATING  AND  COOLING 

Skew 

Documentat i on 

Leaks 

Safety 

SUBSYSTEM 

of  presence 

GENERATION  SOURCE 

Furnace/burner 

1 

1 

- 

1 

Stove 

! 

1 

- 

1 

Heat  pump 

1 

1 

- 

1 

Electrical  heaters 

1 

1 

- 

1 

A i r- cond i t i oner 

1 

1 

- 

1 

Rad i ators 

1 

1 

- 

1 

Sol ar  col  lector 

! 

! 

- 

1 

F i rep l ace 

1 

1 

- 

1 

Fasteners 

~ 

1 

- 

1 

TRANSPORT 

Ductwork 

1 

1 

2 

- 

Piping 

! 

1 

2 

- 

Restra i n  ts 

- 

1 

- 

1 

Fasteners 

- 

1 

- 

1 

Conduct ion/convcc  t ion 

' 

> 

- 

■' 

STORAGE 

Tank 

t 
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2 

- 

Mai  1 

1 

1 

- 

- 

Rockbed 

1 

1 

2 

- 

Other 

“ 

1 

~ 

- 

OTHER 

Ch i mnoy 

1 

1 

- 

1 

Ch i mney  1 i ners 

1 

1 

- 

1 
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Table  6 

Applications  of  the  Sequential  Construction  Analyzer 
in  Construction  Sites 


SITE  ELEMENTS 


BELOW  GRADE 


GROUND  LEVEL 


ABOVE  GRADE 


PROPERTIES  OR  ATTRIBUTES' 


TOPOGRAPHICAL  PROPERTIES 


Slope  Depth  Height  Width 


1  1 


OTHER  PROPERTIES 


Vegetation  Area 


'Numbers  refer  to  the  most  likely  method  for  generating  the  image 
to  be  analyzed  by  the  SCA:  1 -video  camera;  2-thermographic 


Figure  1. 


Results  of  a  field  survey  of 
struction  analyzer. 
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Quality  Assurance  Applications  of  the  Sequential  Construction 

Analyzer  in  Paving 


4  DEVELOPMENT  PROCESS  FOR  TYPICAL  SCA  APPLICATION:  OVERVIEW 


Numerical  Complexities 

The  most  significant  factor  that  determines  if  an  application  can  be  implemented  is 
the  number  of  calculations  needed  to  complete  the  task.  If  this  number  becomes  too 
large,  then  a  computer  may  not  be  able  to  finish  the  task  in  an  acceptable  period  of 
time.  At  first,  this  limitation  may  seem  strange.  It  can  be  explained  in  two  ways. 

When  such  a  linear  computer  has  an  image-processing  task,  it  makes  calculations  on 
a  two-dimensional  data  domain.  As  the  machine  performs  more  intricate  calculations  on 
this  two-dimensional  domain,  the  task  complexity  increases  by  the  square  of  difficulty  of 
the  task.  For  example,  if  a  program  makes  calculations  on  a  picture  which  is  a  grid  of 
256  by  256  pixels  and  then  increases  the  grid  size  to  512  by  512  pixels,  the  program 
executes  four  times  as  many  calculations  rather  than  twice  as  many.  By  taking  into 
account  that  the  image  is  a  two-dimensional  representation  (a  picture)  of  an  event  that 
occurs  in  three  dimensions  (construction),  the  complexity  increases  further. 

Another  way  of  describing  why  the  number  of  calculations  can  become  overwhelm¬ 
ing  is  to  think  of  the  system  as  a  stimulus/response  mechanism.  The  following  task 
illustrates  the  stimulus/response  idea.  The  domain  consists  of  two  cards,  one  with  a 
black  square  and  one  with  a  black  triangle,  which  can  be  held  one  at  a  time  in  a  standard 
position  and  with  standard  lighting  before  a  camera. 

The  system  is  tasked  with  recognizing  which  card  is  being  photographed.  This  task 
is  easy  because  the  system  has  to  differentiate  between  only  two  stimuli  that  can  be  pre¬ 
sented.  In  a  more  complicated  problem,  the  computer  is  required  to  distinguish  a  card 
held  in  any  orientation  as  long  as  it  is  in  focus  and  square  to  the  camera.  Assume  that 
the  computer  can  resolve  this  rotation  to  within  one  part  in  1000.  Now  it  must  distin¬ 
guish  1000  stimuli  out  of  2000  (2  cards  x  1000)  allowable  stimuli.  This  problem  is  much 
more  difficult  than  the  original  one.  If  changes  are  allowed  in  the  size  of  the  square,  the 
length  of  the  sides  of  the  triangle,  lighting,  etc.,  the  problem  grows  more  complicated. 
If  the  stimulus  is  not  limited  to  either  a  square  or  a  triangle,  the  number  of  allowable 
stimuli  becomes  astronomical  (e.g.,  with  a  field  of  512  by  480  pixels,  each  with  8  bits  of 
information,  the  number  of  allowable  stimuli  is  2(480  x  5,^x  8)  =  21966080  =  iO591849. 

The  above  description  of  the  numerical  complexities  that  may  be  incurred  when 
deriving  information  via  the  SCA  appear  mind-boggling  and  impractical.  However, 
current  technology  is  at  a  state  that  will  allow  many  applications  to  be  achieved  in  real 
time.  Applications  that  may  benefit  from  the  SCA  have  to  be  identified  and  then, 
through  development,  productive  enhancement  devices  made;  these  applications  are 
possible  now. 


Two  Different  Development  Strategies 

Once  a  problem  is  analyzed  as  being  computationally  manageable,  two  main 
circumstances  determine  the  development  strategy  of  a  given  SCA  application: 

1.  There  is  no  complete  set  of  image-processing  algorithms,  i.e.,  a  particular 
application  may  or  may  not  be  accomplished  by  simply  stringing  together  a  combination 
of  existing  algorithms. 


2.  For  any  particuiar  set  of  algorithms  that  comprise  an  application,  it  is  difficult 
(if  not  impossible)  to  calculate  the  anticipated  experimental  error  from  a  purely  theore¬ 
tical  perspective.  SCA  applications  take  place  under  harsh,  poorly  controlled  environ¬ 
mental  conditions.  The  errors  introduced  by  such  conditions  include:  nonunlforin  light 
ing  for  which  the  system  may  have  difficulty  compensating,  extreme  temperatures  that 
cause  the  cameras  to  behave  differently,  dust  and  other  optical  interference,  short 
exposure  times  due  to  movement  which  lower  the  signal-to-noise  ratio,  and  physical 
punishment  from  weather  and  mechanical  abuse.  These  environmental  errors  are  ex¬ 
tremely  large  compared  with  algorithm-induced  error.  Error,  therefore,  can  only  be 
determined  experimentally. 

Because  of  these  factors,  the  implementation  strategy  tends  to  follow  one  of  two 
paths.  The  first  path  occurs  when  there  are  known  image-processing  techniques  suffi¬ 
cient  to  perform  the  task  with  good  return  on  investment.  In  this  case,  a  program  would 
be  prepared  to  run  one  or  several  types  of  analysis,  place  the  unit  in  the  field,  and  with 
the  help  of  a  human,  run  enough  trials  to  determine  which  method  is  best  and  if  this  best 
method  is  good  enough.  The  unit  would  then  be  modified  to  use  the  best  method  resulting 
in  a  finished  product. 

The  second  development  path  occurs  when  the  correct  image-processing  techniques 
are  not  known  and  may  not  be  found.  When  this  occurs,  it  is  best  to  start  with  a  human- 
guided  system.  Care  must  be  taken  to  ensure  that  the  human-guided  system  yields  a 
good  return  on  investment  because  there  is  no  guarantee  that  progress  can  be  made 
toward  an  automated  system.  As  algorithms  are  developed  that  can  handle  tasks  and 
task  components,  they  are  added  to  the  system,  either  relieving  the  human  of  them  or 
using  them  as  a  first  approximation  to  completing  the  task  and  allowing  the  human  to 
edit  the  results.  In  this  way,  the  machine  eventually  can  become  more  and  more  inde¬ 
pendent  of  human  intervention,  with  the  ability  to  do  analyses  autonomously. 


Example  of  the  First  Development  Strategy? 


The  example  involves  truck  identification  at  a  site  entrance.  This  application 
consists  of  setting  a  camera  at  the  side  of  the  road  of  a  construction  site  entrance, 
taxing  pictures  of  each  vehicle  that  passes,  and,  when  the  picture  contains  a  certain  type 
of  vehicle,  creating  a  record  of  the  time  and  vehicle  type. 

Because  the  application  is  restricted  to  a  small  number  of  vehicle  types,  there  are 
several  image-processing  strategies  that  might  work.  This  application,  therefore,  should 
be  developed  using  the  first  development  strategy. 


First,  pictures  of  the  particular  vehicles  of  interest  are  taken  from  the  same  angle 
at  which  the  camera  will  view  them  onsite.  These  reference  photos  are  analyzed  by 
looking  at  the  angles  of  the  leading  edges  on  the  front  end  of  the  truck.  Histograms  of 
pixel  counts  as  a  function  of  angle  are  recorded  and  stored. 


Environmental  variables  are  diminished  by  using  three  light  beam/photo  cell  pairs 
at  10-ft  intervals  across  the  roadway  at  the  construction  site  entrance.  If  none  of  the 
beams  are  broken,  the  system  takes  a  snapshot  every  few  seconds  as  a  reference  back¬ 
ground.  If  one  of  the  two  beams  at  either  end  is  broken,  the  system  assumes  something  is 
coming  and  retains  the  previous  background  shot.  When  the  center  beam  is  broken,  the 
system  concludv-s  that  it  is  looking  at  an  item  of  interest  and  takes  a  snapshot. 


The  background  picture  is  compared  with  the  snapshot  to  isolate  those  pixels  that 
actually  form  the  object  which  broke  the  beam.  The  angles  on  the  leading  edge  of  the 
object's  picture  are  then  placed  in  a  histogram  (of  the  same  type  made  for  the  reference 
vehicles)  and  compared  with  the  reference  histograms  of  the  truck  reference  photos. 
The  closest  match  wins.  If  none  of  the  reference  histograms  match  well,  the  system 
assumes  the  object  is  something  else  and  does  not  count  it. 


Primitive  versions  of  this  application  have  been  developed  and  tested  to  identify 
and  count  trucks  on  a  construction  site.  This  work  was  performed  by  O'Brien  in  conjunc¬ 
tion  with  the  University  of  New  South  Wales.' 


Example  of  the  Second  Development  Strategy 


The  example  here  is  for  automated  pavement  analysis.  This  system,  which  is 
currently  under  development  at  USA-CERL,  is  used  to  analyze  pictures  of  pavement 
surfaces  and  forward  the  resulting  analysis  to  another  program  called  PAVER. 


PAVER  is  a  field-tested  and  validated  pavement  maintenance  management  system 
for  use  by  military  installations,  USACE  Districts,  airports,  and  civilian  municipalities. 
PAVER  is  designed  to  optimize  the  allocation  of  funds  for  pavement  maintenance  and 
repair.  PAVER'S  key  component  is  a  Pavement  Condition  Index  (PCI).  The  PCI  is  an 
objective  rating  of  pavement  condition  based  on  observed  distress  types  (cracking, 
bumps,  sags,  etc.).  The  PCI  provides  a  consistent  measure  of  a  pavement's  structural 
integrity  and  operational  condition. 


Information  about  a  pavement's  condition  is  usually  gathered  by  someone  who  has 
to  view  the  pavement,  determine  which  conditions  are  present,  carefully  measure  the 
size  of  the  area  affected  and  record  all  information  for  later  use  by  PAVER. 


The  automated  pavement  evaluation  system  provides  a  way  for  users  to  cut  down 
this  workload.  The  pavement  sections  are  photographed  and  the  photos  fed  to  an  image- 
processing  system.  The  user  works  interactively  with  the  system  to  identify  pavement 
distresses.  All  other  work  is  left  to  the  computer. 


In  this  application,  environmental  factors  are  eliminated  by  taking  the  pavement 
photographs  from  a  camera  with  a  flash  unit  mounted  in  the  roof  of  a  van  with  no  floor. 
The  van  is  constructed  to  completely  shade  the  pavement  being  photographed.  In  this 
way,  problems  from  inconsistent  lighting  are  eliminated. 


The  image-processing  system  appears  to  the  user  almost  like  a  coloring  book.  The 
user  views  a  photograph  of  a  pavement  section  on  a  video  monitor,  tells  the  system  what 
type  of  pavement  distress  appears  in  the  picture,  and  marks  the  distress  (on  the  video 
screen)  with  a  pointing  device  (i.e.,  a  mouse  or  a  light  pen).  The  system  performs  all 
numerical  calculations  and  automatically  forwards  this  information  to  PAVER.  The  user 
makes  no  numerical  measurements,  performs  no  numerical  calculations,  and  needs  no 
manual  numerical  recordkeeping. 


'j.  B.  O'Brien,  J.  L.  Knott,  and  R.  W.  Woodhead,  "A  System  for  Fully  Automated  Capture 
and  Analysis  of  Field  Performance  Data,"  Organizing  and  Managing  Construction, 
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Because  the  user  is  relieved  of  numerical  tallying  and  recordkeeping,  the  system 
saves  labor  and  is  of  immediate  economic  benefit.  Development  of  various  methods  for 
automatic  detection  of  pavement  distresses  is  under  way  and  will  be  added  to  the  pack¬ 
age  as  each  becomes  available.  Even  if  no  suitable  methods  are  found,  the  system  is  a 
good  investment  for  its  users.  This  is  an  example  of  the  second  development  path  be¬ 
cause  a  hand-guided  system  is  justified  by  return  on  investment  and  any  autonomous 
analysis  by  the  system  is  helpful  but  not  crucial. 
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5  CONCLUSIONS  AND  RECOMMENDATIONS 


This  study  has  identified  the  capabilities  and  potential  applications  of  image-pro¬ 
cessing  technology  for  military  construction.  It  is  clear  that  there  are  many  potential 
applications  for  an  SCA  that  use  this  technology.  Recent  advances  in  the  use  of  compu¬ 
ters  for  image  processing  hold  much  promise  for  construction  applications  such  as  quality 
assurance,  tracking  construction  progress,  and  obtaining  data  for  maintenance  decision¬ 
making. 

The  image-processing  concept  has  been  explained  briefly.  Some  of  the  mathemati¬ 
cal  algorithms  have  been  presented  as  having  the  highest  potential  for  use  in 
construction. 

This  report  has  described  some  potential  SCA  applications  in  typical  building  con¬ 
struction  processes.  However,  opportunities  for  the  use  of  an  SCA  are  also  replete  in 
construction  areas  such  as  dams,  flood  plan  surveys,  bridge  construction,  and  other  points 
of  interest  to  USACE.  The  tables  in  this  report  suggest  an  approach  by  which  other 
opportunities  for  use  of  the  SCA  can  be  identified;  that  is,  by  separating  construction 
processes  into  smaller  subelements,  it  is  possible  to  assess  the  individual  components  for 
SCA  applicability. 

To  maximize  the  potential  of  the  SCA,  these  follow-on  activities  are  recom¬ 
mended: 


•  Implement  a  demonstration  program  to  evaluate  use  of  the  SCA  in  construction 
applications 

•  Develop  guidelines  and  criteria  for  using  the  SCA  in  construction  QA  applica¬ 
tions 

•  Identify  additional  opportunities  for  use  of  SCA  in  construction  applications  not 
addressed  in  this  report  for  development  by  USACE 

•  Present  information  on  SCA  for  contractors  as  a  potential  contractor  QC  tool. 
Contractors  could  use  the  SCA  to  help  prevent  deficiencies  and  tear  them  out 
rather  than  locate  them  and  effect  corrective  action  as  would  be  the  case  if  he 
SCA  is  used  for  QA. 
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